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1. A hierarchical clustered parallel processing system comprising: 

at least one cluster of computer processing system forming a node of a 
hierarchical cluster, each cluster of computer processing systems 
comprising: 

a plurality of computer systems designated to be members of nodes 
of said cluster; 

a physical network connected to allow each computer system of the 
plurality of computer systems to transfer data between any of 
the plurality of computer systems; 

a virtual multicast bus to designate communication between 
member computer systems; and 

a configuration service apparatus in communication with each of the 
computer systems to provide each of the plurality of computer 
systems with: 

a node identification to identify a node for each member 
computer system within said cluster, 

a multicast bus address to broadcast communications to 
said members of said cluster by way of said virtual 
cluster bus, and 
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a node priority list designating a priority for each node within 
said cluster; and 

a cluster supervising processor to provide operational control 
services for said cluster, said cluster supervising processor 
being selected of said member computer systems according to 
the priority from said priority list. 

2. The hierarchical clustered parallel processing system of claim 1 wherein the 
configuration service apparatus further provides a disk access list. 

3. The configuration service apparatus of claim 2 wherein the disk access list 
comprises identification of accessible disks, disk mount points, and failure 
detection locations. 

4. The hierarchical clustered parallel processing system of claim 1 wherein the 
cluster supervising processor maintains: 

a cluster topology table detailing connectivity for each node of the cluster 
and a disk access status for each disk within said cluster; 

a disk usage table describing current capacity and loading for each disk 
within said cluster; 

a node usage table describing a streaming capacity for each node of said 
cluster and a current loading for each node of said cluster; and 
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a cluster map describing network addresses for each of a plurality of 
servers in communication with said cluster and listing of nodes within 
said cluster, network addresses for said nodes, and an operational 
status of said nodes. 

5. The hierarchical clustered parallel processing system of claim 1 wherein a group 
of said member computer systems of said cluster are configured as a sub- 
cluster, said sub-cluster being a node of said cluster. 

6. The hierarchical clustered parallel processing system of claim 4 wherein the each 
cluster of computer processing systems further comprises a fault detection 
apparatus within each member computer system: 

to periodically receive a first processor status message from a first 
adjacent node; 

to append a second processor status message of a current node to said 
first processor status message; and 

to periodically transmit said first and second processor status message to 
a second adjacent node. 

7. The hierarchical clustered parallel processing system of claim 6 wherein said 
cluster supervising processor receives an accumulation of the processor status 
messages from all nodes of said cluster. 
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8. The hierarchical clustered parallel processing system of claim 6 wherein, if the 
fault detection apparatus does not receive said first processor status message for 
a number of periods, said first adjacent node is declared to have failed and a 
failure declaration is appended to said second processor status message. 

9. The hierarchical clustered parallel processing system of claim 8 wherein, upon 
receipt of said failure declaration, the cluster supervising processor modifies said 
cluster map to reflect failure of the node. 

10. The hierarchical clustered parallel processing system of claim 4 wherein the 
cluster supervising processor periodically posts a supervisor notification 
message on said virtual multicast bus, said supervisor notification message 
comprises a node identification and a network address for said cluster 
supervising processor. 

1 1 . The hierarchical clustered parallel processing system of claim 1 0 wherein the 
supervisor notification message further comprises the cluster topology and a 
current cluster map. 

12. The hierarchical clustered parallel processing system of claim 1 0 wherein, if one 
node of cluster does not receive said supervisor notification message within a 
notification time, said node becomes said cluster supervising processor, updates 
said cluster topology table and said cluster map, transmits a cluster supervising 
processor update message, and the supervisor notification message. 

13. The hierarchical clustered parallel processing system of claim 4 wherein: 
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each node of said cluster periodically determines whether each disk to 
which said node has access is functioning and if any disk is not 
functioning; 

the node creates a disk failure message for the disk not functioning for 
transfer to an adjacent node; 

wherein said adjacent node transfers said disk failure node to subsequent 
adjacent nodes until said cluster supervising processor receives said 
disk failure message; 

wherein upon receipt of multiple disk failure messages from multiple 
nodes for the disk not functioning, the cluster supervising processor 
declares a disk failure, updates the disk usage table, and reassigns all 
the transfer of video data files from a failing node to an active node. 

14. The hierarchical clustered parallel processing system of claim 10 wherein a new 
node joins said cluster by the steps of: 

listening to said virtual multicast bus for a supervisor notification message 
from the present cluster supervising processor; 

posting on said virtual multicast bus a join request message providing a 
node identification, a network address for said node, and a disk access 
list for said node; 
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updating by the present cluster supervising processor the cluster map and 
the cluster topology; and 

placing a new supervisor notification message upon said virtual multicast 
bus including said new node. 

15. The hierarchical clustered parallel processing system of claim 14 wherein the 
new node joins said cluster further by the step of: 

ceasing posting on said virtual multicast bus said join request message. 

16. The hierarchical clustered parallel processing system of claim 14 wherein the 
new node becomes the cluster supervising processor, if said new node has a 
priority that supercedes said present cluster supervising processor. 

17. The hierarchical clustered parallel processing system of claim 16 wherein the 
new node acting as the present cluster supervising processor transmits the 
supervisor notification message and the original cluster supervising processor 
ceases transmitting said supervisor notification message. 

18. The hierarchical clustered parallel processing system of claim 17 wherein if the 
new node does not transmit the supervisor notification message by the 
notification time, the original cluster supervising processor resumes transmission 
of the supervisor notification message. 

19. The hierarchical clustered parallel processing system of claim 10 wherein a node 
leaves a cluster by the steps of: 
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posting a leave message on said virtual multicast bus, said leave 
message containing the node identification and the network address 
for said node; 

updating by the cluster supervising processor of the cluster map and the 
cluster topology; and 

posting on the virtual multicast bus the supervisor notification message 
with the updated cluster map and cluster topology. 

20. The hierarchical clustered parallel processing system of claim 19 wherein the 
node leaving the cluster ceases posting the leave message upon receipt of the 
supervisor notification message with the updated cluster map and cluster 
topology. 

21 . The hierarchical clustered parallel processing system of claim 1 9 wherein if the 
node leaving the cluster is the cluster supervising processor, the node of the 
cluster of the priority list then becomes the cluster supervising processor. 

22. The hierarchical clustered parallel processing system of claim 1 wherein said 
cluster is formed and said cluster supervising processor is designated by the 
steps of: 

listening to said virtual multicast bus for a supervisor notification message 
from the cluster supervising processor by each node of the cluster; 
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6 if no supervisor notification message is received, designating each node a 

7 single node cluster of its own; 

8 designating each node the cluster supervising processor of its single node 

9 cluster; 

10 transmitting by each cluster supervising processor of each single node 

11 cluster the supervisor notification message for each single node 

12 cluster; 

13 ceasing by those nodes having a lower priority from transmitting 

14 supervisor notification messages such that the node with a highest 
is priority is the cluster supervising processor; and 

16 joining said cluster by those nodes with lower priority by posting on said 

17 virtual multicast bus a join request message providing a node 

is identification, a network address for said node, and a disk access list 

19 for said node. 

1 23. A cluster of computer processing systems comprising: 

2 a plurality of computer systems designated to be members of nodes of 

3 said cluster; 

4 a physical network connected to allow each computer system of the 

5 plurality of computer systems to transfer data between any of the 

6 plurality of computer systems; 
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a virtual multicast bus to provide communication between member 
computer systems; and 

a configuration sen/ice apparatus in communication with each of the 
computer systems to provide each of the plurality of computer systems 
with: 

a node identification to identify a node for each member 
computer system within said cluster, 

a multicast bus address to broadcast communications to said 
members of said cluster by way of said virtual cluster bus, 
and 

a node priority list designating a priority for each node within 
said cluster; and 

a cluster supervising processor to provide operational control services for 
said cluster, said cluster supervising processor being selected of said 
member computer systems according to the priority from said priority 
list. 

24. The cluster of computer processing systems of claim 23 wherein the 
configuration service apparatus further provides a disk access list. 
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25. The configuration service apparatus of claim 24 wherein the disk access list 
comprises identification of accessible disks, disk mount points, and failure 
detection locations. 

26. The cluster of computer processing systems of claim 23 wherein the cluster 
supervising processor maintains: 

a cluster topology table detailing connectivity for each node of the cluster 
and a disk access status for each disk within said cluster; 

a disk usage table describing current capacity and loading for each disk 
within said cluster; 

a node usage table describing a streaming capacity for each node of said 
cluster and a current loading for each node of said cluster; and 

a cluster map describing network addresses for each of a plurality of 
servers in communication with said cluster and listing of nodes within 
said cluster, network addresses for said nodes, and an operational 
status of said nodes. 

27. The cluster of computer processing systems of claim 23 wherein a group of said 
member computer systems of said cluster are configured as a sub-cluster, said 
sub-cluster being a node of said cluster. 

28. The cluster of computer processing systems of claim 26 further comprising a fault 

detection apparatus within each member computer system: 
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to periodically receive a first processor status message from a first 
adjacent node and transmit; 



to append a second processor status message of a current node to said 
first processor status message; and 

to periodically transmit said first and second processor status message to 
a second adjacent node. 

29. The cluster of computer processing systems of claim 28 wherein said cluster 
supervising processor receives an accumulation of the processor status 
messages from all nodes of said cluster. 

30. The cluster of computer processing systems of claim 28 wherein, if the fault 
detection apparatus does not receive said first processor status message for a 
number of periods, said first adjacent node is declared to have failed and a 
failure declaration is appended to said second processor status message. 

31 . The cluster of computer processing systems of claim 30 wherein, upon receipt of 
said failure declaration, the cluster supervising processor modifies said cluster 
map to reflect failure of the node. 

32. The cluster of computer processing systems of claim 26 wherein the cluster 
supervising processor periodically posts a supervisor notification message on 
said virtual multicast bus, said supervisor notification message comprises a node 
identification and a network address for said cluster supervising processor. 
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33. The cluster of computer processing systems of claim 32 wherein the supervisor 
notification message further comprises the cluster topology and a current cluster 
map. 

34. The cluster of computer processing systems of claim 32 wherein, if one node of 
cluster does not receive said supervisor notification message within a notification 
time, said node becomes said cluster supervising processor, updates said cluster 
topology table and said cluster map, transmits a cluster supervising processor 
update message, and the supervisor notification message. 

35. The cluster of computer processing systems of claim 26 wherein: 

each node of said cluster periodically determines whether each disk to 
which said node has access is functioning and if any disk is not 
functioning; 

the node creates a disk failure message for the disk not functioning for 
transfer to an adjacent node; 

wherein said adjacent node transfers said disk failure node to subsequent 
adjacent nodes until said cluster supervising processor receives said 
disk failure message; 

wherein upon receipt of multiple disk failure messages from multiple 
nodes for the disk not functioning, the cluster supervising processor 
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declares a disk failure, updates the disk usage table, and reassigns all 
the transfer of video data files from a failing node to an active node. 

36. The cluster of computer processing systems of claim 32 wherein a new node 
joins said cluster by the steps of: 

listening to said virtual multicast bus for a supervisor notification message 
from the present cluster supervising processor; 

posting on said virtual multicast bus a join request message providing a 
node identification, a network address for said node, and a disk access 
list for said node; 

updating by the present cluster supervising processor the cluster map and 
the cluster topology; and 

placing a new supervisor notification message upon said virtual multicast 
bus including said new node. 

37. The cluster of computer processing systems of claim 36 wherein the new node 
joins said cluster further by the steps of: 

ceasing posting on said virtual multicast bus said join request message. 

38. The cluster of computer processing systems of claim 36 wherein the new node 
becomes the cluster supervising processor, if said new node has a priority that 
supercedes said present cluster supervising processor. 
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39. The cluster of computer processing systems of claim 38 wherein the new node 
acting as the present cluster supervising processor transmits the supervisor 
notification message and the original cluster supervising processor ceases 
transmitting said supervisor notification message. 

40. The cluster of computer processing systems of claim 39 wherein if the new node 
does not transmit the supervisor notification message by the notification time, the 
original cluster supervising processor resumes transmission of the supervisor 
notification message. 

41 . The cluster of computer processing systems of claim 32 wherein a node leaves a 
cluster by the steps of: 

posting a leave message on said virtual multicast bus, said leave 
message containing the node identification and the network address 
for said node; 

updating by the cluster supervising processor of the cluster map and the 
cluster topology; and 

posting on the virtual multicast bus the supervisor notification message 
with the updated cluster map and cluster topology. 

42. The cluster of computer processing systems of claim 41 wherein the node 
leaving the cluster ceases posting the leave message upon receipt of the 
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supervisor notification message with the updated cluster map and cluster 
topology. 

43. The cluster of computer processing systems of claim 41 wherein if the node 
leaving the cluster is the cluster supervising processor, the node of the cluster of 
the priority list then becomes the cluster supervising processor. 

44. The cluster of computer processing systems of claim 23 wherein said cluster is 
formed and said cluster supervising processor is designated by the steps of: 

listening to said virtual multicast bus for a supervisor notification message 
from the cluster supervising processor by each node of the cluster; 

if no supervisor notification message is received, designating each node a 
single node cluster of its own; 

designating each node the cluster supervising processor of its single node 
cluster; 

transmitting by each cluster supervising processor of each single node 
cluster the supervisor notification message for each single node 
cluster; 

ceasing by those nodes having a lower priority from transmitting 
supervisor notification messages such that the node with a highest 
priority is the cluster supervising processor; and 
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joining said cluster by those nodes with lower priority by posting on said 
virtual multicast bus a join request message providing a node 
identification, a network address for said node, and a disk access list 
for said node. 
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